Smart speaker system

ABSTRACT

A smart speaker system is used as an emergency notification system. The smart speaker has a loudspeaker, a microphone, a visual display, processing capability, and an emergency flasher in the same enclosure. The smart speaker is in communication with a network-connected server providing at least distribution of emergency notification messages to appropriate clients. The amount of audio processing shared between the smart speaker and the server may vary, depending on the requirements of the installation site. Once an emergency voice communication is detected in the microphone output, the system generates a confirmation message that is played over the loudspeaker. Upon detection of an appropriate voice response from the user, the system generates an emergency response message, sends it to the distribution network, and activates the visual display, loudspeaker, and emergency flasher. Noise cancellation circuitry prevents loudspeaker interference with microphone output.

FIELD OF ART

The present invention relates to voice-responsive (smart) speakersystems. The present invention more particularly relates to facilityemergency notification systems using smart speakers.

BACKGROUND OF THE INVENTION

Emergency notification systems play an important role in facilitymanagement. Many such systems rely on alarms initiated at the point ofthe emergency, such as pulling a fire alarm. Such point-of-problemsystems do not automatically cover a broad range of possibleemergencies, such as flood, medical, terrorism, chemical leakage, etc.As a result, the most efficient allocation of emergency responseresources cannot always be made.

SUMMARY OF THE INVENTION

The present invention provides a voice-initiated emergency notificationsystem from an observer of such emergency. The observer's voice into thesmart speaker, which includes a microphone, identifies a location andtype of emergency. A voice recognition system recognizes key words inthe observer's voice message and initiates a confirmation message backto the observer. If the observer verifies the message, then theemergency voice message is sent, via an internet or intranet (network,generally), to the appropriate first responders. Emergency voicemessages can relate to more than one emergency, such as “fire withinjuries.” Audio processing, including voice recognition, is sharedbetween the smart speaker and the server, with the degree varyingresponsive to installation site requirements. Noise cancellationcircuitry prevents loudspeaker interference with microphone output. Thenovel use of emergency confirmation messages and audible responsesthereto, the multi-sensory response of the smart speaker (voice, siren,flashing light, text, symbols), the microphone noise cancellation of theloudspeaker output, and smart client selection based on the emergencynotification message are points of novelty of the present invention.

DESCRIPTION OF THE FIGURES OF THE DRAWINGS

The present invention will hereinafter be described in conjunction withthe following drawing figures, wherein like numerals denote likeelements, and

FIG. 1 is a diagrammatic view illustrating an exemplary embodiment of asmart speaker system showing a first process step, according to apreferred embodiment of the present invention;

FIG. 2 is a diagrammatic view illustrating an exemplary embodiment of asmart speaker system of FIG. 1 showing a second process step, accordingto a preferred embodiment of the present invention;

FIG. 3 is a diagrammatic view illustrating an exemplary embodiment of asmart speaker system of FIG. 1 showing a third process step, accordingto a preferred embodiment of the present invention;

FIG. 4 is process flow chart view illustrating an exemplary embodimentof a smart speaker system of FIG. 1, according to a preferred embodimentof the present invention;

FIG. 5 is process flow chart view illustrating a second exemplaryembodiment of a smart speaker system of FIG. 1, according to a preferredembodiment of the present invention;

FIG. 6 is process flow chart view illustrating a third exemplaryembodiment of a smart speaker system of FIG. 1, according to a preferredembodiment of the present invention; and

FIG. 7 is a diagrammatic view illustrating the exemplary embodiment of asmart speaker system of FIG. 1, according to a preferred embodiment ofthe present invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a diagrammatic view illustrating an exemplary embodiment of asmart speaker system 100 showing a first process step, according to apreferred embodiment of the present invention. A user 102 provides avoice message input 104 to a smart speaker 106. The voice message input104 contains an emergency keyword, an emergency type, and a location ofthe emergency. For example: “Globalcom I see a fire at gate 18”. Where“Globalcom” is a key word indicating an emergency, “fire” is theemergency type, and “gate 18” is the emergency location.

The smart speaker system 100 includes the smart speaker 122 and theserver 116 having a network 118 connection to clients 120. The smartspeaker system 100 is intended for large facility installations.Typically, smart speakers 122 would be mounted in various places in thefacility, much like pull-down fire alarm actuators are today. A smartspeaker may also be located in a centralized security office, whereemergencies may be observed by closed circuit television, or the like.Each smart speaker 122 would be connected to the server 116, eitherdirectly or via a communications network similar to network 118.

Smart speaker 122 includes a housing 106 supporting a microphone 112, avisual display 110, an emergency flasher 108, a loudspeaker 706 (seeFIG. 7) and a communication link 114 to server 116. Voice message input104 is received in microphone 112 and may be processed partially withinthe smart speaker 106 and partially within the server 116, as will bediscussed in more detail below. The server 116 receives either theentire voice message input 104 or a parsed and reformatted versionthereof. The server 116 is connected to the Internet 118 or an intranet118 and thereby to clients 120. Clients 120 are represented as an alarmbell 120, but any of various notification devices may be substituted oradded. For example, flashing lights 108, computer notifications, smartspeakers 122, sirens, emergency marquee floor lights, video screens 110,and the like.

FIG. 2 is a diagrammatic view illustrating an exemplary embodiment of asmart speaker system 100 of FIG. 1 showing a second process step,according to a preferred embodiment of the present invention. Server 116sends 414 (see FIG. 4) a confirmation message to the smart speaker 122via communication link 204 which, in some embodiments, may be the sameas communication link 114. The confirmation message may be, fornon-limiting example, “Did I hear you correctly, do you want to initiatean emergency notification?” The smart speaker 106 provides audibleoutput 202 of the confirmation message to the user 102.

FIG. 3 is a diagrammatic view illustrating an exemplary embodiment of asmart speaker system 100 of FIG. 1 showing a third process step,according to a preferred embodiment of the present invention. User 102verbally confirms the confirmation message with a voice message input104. The confirmation is processed either in the smart speaker 122 andsent 410 (see FIG. 4) to the server 116 on communication link 114 or, inother embodiments, is sent to the server 116 via link 114 and processedin the server 116. The confirmation response and original emergencymessage is further processed in the server 116 and emergencynotifications are sent out to the smart speaker 1122 on communicationlink 204 and to the Internet 118 on communication link 304. The Internet118 distributes alarm messages to clients 120 via communication link 306and the clients 120 activate their respective responses 308. The smartspeaker 122 activates an audible alarm 302 through the loudspeaker 706(see FIG. 7), the emergency flasher 108, and provides relevant alarminformation on the visual display 110 as either text, symbols, or both.Preferably, the microphone 112 is connected internally to noisecancelation circuitry that nulls out the audible alarm 302, therebyenabling further voice message input 104. The smart speaker system 100is capable of concurrently handling multiple emergencies. For example,fire and medical emergencies may arise concurrently.

FIG. 4 is process flow chart view illustrating an exemplary embodimentof a smart speaker system 100 of FIG. 1, according to a preferredembodiment of the present invention. In this embodiment, audioprocessing is shared between the smart speaker 122 and the server 116.When the smart speaker system 100 is on, it constantly monitors 402 themicrophone output 428 for keywords, such as the example given above:“Globalcom”. The smart speaker 122 may be programmed to have a varietyof keywords for a state of emergency, type of emergency, and, in someembodiments, location of emergency (such as “gate”), as well asconfirmation keywords. Keywords may be provided in more than onelanguage. A keyword database 712 (see FIG. 7) is maintained within thesmart speaker system 100. If 404, no keyword is detected, the smartspeaker 122 continues to monitor 402 microphone output 428. If 404 akeyword is detected, the smart speaker system 100 begins recording 406the microphone output 428. If 408 silence ensues for a predeterminedtime, the smart speaker system 100 returns to monitoring 402 microphoneoutput 428. If 408 the voice message continues, the recorded audio andkeywords are sent 410 to the server 116 over communication link 114.Server 116 processes 412 the audio and keywords to produce or select anappropriate confirmation message. The server 116 then sends 414 theconfirmation message to the smart speaker 122 via communication link204. The confirmation message is then delivered audibly via audibleoutput 202 to user 102. If 418, the user 102 does not audibly confirmthe confirmation message, the smart speaker system 100 returns tomonitoring 402 microphone output 428 for keywords. If 418, the user 102does audibly confirm the confirmation message, then the smart speaker122 initiates 420 a transmit session in response to both the voiceconfirmation and the server request 416. Next the smart speaker 122transmits 422 the recorded voice audio to the server 116. The server 116receives and processes 424 the voice audio to produce and determine thedestinations for an emergency notification. Finally, the server 116initiates 426 an emergency notification over communication link 304 tonetwork-connected destinations and over communication link 204 to thesmart speaker 122.

FIG. 5 is process flow chart view illustrating a second exemplaryembodiment of a smart speaker system 500, according to a preferredembodiment of the present invention. In this embodiment, the smartspeaker 502 does most of the audio processing. When the smart speakersystem 500 is on, it constantly monitors 402 the microphone 112 outputfor keywords, such as the example given above: “Globalcom”. The smartspeaker 502, which is a configuration of smart speaker 122, may beprogrammed to have a variety of keywords for a state of emergency, typeof emergency, and, in some embodiments, location of emergency (such as“gate”). Keywords may be provided in more than one language. If 404, nokeyword is detected, the smart speaker 502 continues to monitor 402microphone output 428. If 404 a keyword is detected, the smart speakersystem 500 begins recording 406 the voice message input 104. If 408silence ensues for a predetermined time, the smart speaker system 500ends 504 the recording and returns to monitoring 402 microphone output428. If 408 the voice message continues, the smart speaker 502 parses506 the voice message to produce or select a confirmation message. Thesmart speaker 502 then sends the confirmation message to the user 102via audible output 202. If 418, the user 102 does not audibly confirmthe confirmation message, the smart speaker system 500 returns tomonitoring 402 microphone output 428 for keywords. If 418, the user 102does audibly confirm the confirmation message, then the smart speaker502 transmits 508 the parsed audio to the server 116. The server 116initiates 510 an emergency notification by producing or selecting theemergency notification content and selecting notification clients 120responsive the parsed audio and then transmitting the emergencynotifications to the selected clients 120 over communication link 304and to the smart speaker 502 via communication link 204.

FIG. 6 is process flow chart view illustrating a third exemplaryembodiment of a smart speaker system 600 of FIG. 1, according to apreferred embodiment of the present invention. In this embodiment, theserver 116 does most of the audio processing. Smart speaker 602 is aconfiguration of smart speaker 122. When the smart speaker system 600 ison, received voice message input 104 at the smart speaker 602 from user102 is communicated 604 to the server 116 via communication link 114.The server 116 processes 606 the audio, including detecting keywords,parsing the audio, and producing or selecting a confirmation messageresponsive to the received audio. If 608 does not detect an emergencyfrom a keyword, the smart speaker 602 continues to send 604 voice audioto the server 116. If 608 does detect an emergency from a keyword, theserver 116 communicates 610 the confirmation audio to the smart speaker602. The smart speaker 602 then communicates the confirmation message tothe user 102 via audible output 202. If 418, the user 102 does notaudibly confirm the confirmation message, the smart speaker system 600returns to communicating 604 microphone output 428 to the server 116 forkeyword detection. If 418, the user 102 does audibly confirm theconfirmation message, then the smart speaker 602 transmits 422 the voiceaudio to the server 116. Server 116 receives and processes 424 the voiceaudio to produce an emergency notification and select destinations forthe emergency notification. The server 116 then initiates 426 anemergency notification over communication link 304 and communicationlink 204 to the smart speaker 602.

FIG. 7 is a diagrammatic view illustrating the exemplary embodiment of asmart speaker system 500 of FIG. 5, according to a preferred embodimentof the present invention. Smart speaker 122 includes loudspeaker 706,microphone 112, emergency flasher 108, visual display 110, aprocessor/driver 716, a communication module 702, and an audio processorsubsystem 704. Audio processor subsystem 704 includes a voicerecognition system 708, a keyword database 712, a comparator 710, avoice recorder 714, and a confirmation message generator 718. Inoperation, the microphone 112 output 428 is supplied to the voicerecognition system 708 and made available via communication channel 722to the voice recorder 714. The voice recognition system 708 parses thevoice message input 104 into words which are supplied to the comparator710 via communication channel 724. The comparator 710 is incommunication, via communications channel 740, with the keyword database712 and compares parsed audio with keywords in the keyword database 712.If emergency keywords are found by the comparator 710, the comparator710 triggers the voice recorder 714 to begin recording 406 the voiceaudio. If comparator 710 does not receive voice audio for parsing for apredetermined amount of time, comparator 710 triggers the voice recorder714 to stop recording. If the comparator 710 detects an emergency keyword, the emergency keyword is sent, via communication channel 726, toconfirmation message generator 718, which generates an appropriateconfirmation message and provides audible output 202 of the confirmationmessage through loudspeaker 706. Confirmation messages may bepredetermined or may be constructed by concatenation.

The voice recognition section 708 then parses confirmation microphoneoutput 428 for confirmation keywords. If 418, there is no confirmationkeyword within a predetermined amount of time, comparator 710 continuesto monitor 402 microphone output 428. If 418 a confirmation keyword isdetected, comparator 710 triggers the voice recorder 714 to send therecorded voice message to the communications module 702 overcommunications channel 730.

Communications module 702 communicates the recorded voice message toserver 116 via communications link 114.

Comparator 710 also sends the parsed keywords to the processor/driver716 via communications channel 728. Processor/driver 716 responds to thereceived keywords by generating and sending an emergency notificationmessage to the communications module 702 via communication channel 732.Communications module 702 then sends the emergency notification messageto the server 116 via communications link 114 and server 116 distributesthe emergency notification message to clients 120 via the network 118.Processor/driver 716 also responds to the received keywords to drivevisual display 110 over communications channel 734. Visual display 110presents either text or symbolic representations of the type ofemergency reported. Processor/driver 716 also responds to the receivedkeywords to drive emergency flasher 108 over communications channel 736.Emergency flasher 108 may have RGB LEDs enabling flashing that is colorcoded to the type of emergency. In addition, the pattern, color, andintensity of flashing may correspond to the type of emergency.Processor/driver 716 also responds to the received keywords to driveloudspeaker 706 with an audio signal over communications channel 738.The loudspeaker 706 may be driven to provide variousemergency-responsive audible siren-type alarms or may make linguisticannouncement of the emergency notification message. In a particularembodiment, the loudspeaker 706 may carry live voice from a centralsecurity office.

Those of skill in the art, enlightened by the present disclosure, willappreciate various embodiments of a smart speaker 122 that can producethe above-mentioned outputs from the above-mentioned inputs, all ofwhich embodiments are within the scope of the present invention. In someembodiments, for non-limiting example, processor/driver 716 may bewithin the audio processing subsystem 704.

Embodiments 100, 500, and 600 are preferred embodiments. A morepreferred embodiment is embodiment 100 that can be reconfigured intoembodiment 500 or 600 by hardware reconfiguration, such as swapping outcircuit boards, changing jumper connections, flipping DIP switches, etc.The most preferred embodiment is embodiment 100 that can be configuredinto embodiment 500 or 600 with software installations in the smartspeaker 122 and/or in the server 116.

The novel use of emergency confirmation messages and verbal responsesthereto, the multi-sensory response of the smart speaker 122 (voice,siren, flashing light, text, symbols), the noise cancellation of theloudspeaker output, and smart client selection based on the emergencynotification message are points of novelty of the present invention.

In the claims below, functional claiming is used and there are nostatements of intended use.

We claim:
 1. A smart speaker system comprising: a. a speaker enclosure;b. a loudspeaker supported at least partially within said speakerenclosure c. a microphone supported within said enclosure; d. a serveroperable to communicate over a communication network; e. a communicationlink from said speaker enclosure to said server; f. a visual displaysupported at least partially within said enclosure; and g. an emergencyflasher supported at least partially within said enclosure.
 2. The smartspeaker system of claim 1, comprising: a. a keyword database ofemergency keywords and confirmation keywords; and b. an audio processorsubsystem operable to detect emergency key words, present in saidkeyword database, in an output of said microphone; c. said audioprocessor subsystem operable to detect confirmation key words, presentin said keyword database, in an output of said microphone.
 3. The smartspeaker system of claim 2, operable to record output of said microphoneresponsive to detection of particular said emergency keywords.
 4. Thesmart speaker system of claim 2, wherein said audio processor subsystemis operable to: a. generate a confirmation message responsive todetection of particular said emergency keywords; and b. send an audiosignal of said confirmation message to said loudspeaker.
 5. The smartspeaker system of claim 4, wherein said audio processor subsystem isoperable to: a. detect confirmation keywords in said output of saidmicrophone; and b. communicate said confirmation keywords to aprocessor/driver.
 6. The smart speaker system of claim 4, wherein saidaudio processor subsystem located in at least one of said speakerenclosure and said server.
 7. The smart speaker system of claim 4,wherein said processor/driver is operable, responsive to receivingcommunication of said confirmation keyword and said emergency keywords,to: a. generate an emergency notification message; b. determineappropriate clients to receive said emergency notification message; c.communicate said emergency notification message via said server; d.drive said visual display responsive to reception of said emergencynotification message; e. drive said emergency flasher responsive toreception of said emergency notification message; and f. drive saidloudspeaker responsive to reception of said emergency notificationmessage.
 8. The smart speaker system of claim 7, wherein said server isoperable to distribute said emergency notification message to saidappropriate clients via a communications network.
 9. The smart speakersystem of claim 7, wherein: a. said visual display is operable todisplay at least one of text and symbolic images; b. said emergencyflasher is controllable to modulate intensity, color, and flash pattern,responsive to a type of emergency represented by said emergencynotification message; c. said loudspeaker is operable to audibly presentat least one of the emergency notification message and an alarm sound.10. The smart speaker system of claim 2, comprising noise cancellationcircuitry operable to cancel loudspeaker output in said microphoneoutput.
 11. A smart speaker system comprising: a. a speaker enclosure;b. a loudspeaker supported at least partially within said speakerenclosure c. a microphone supported within said enclosure; d. a serveroperable to communicate over a communication. e. a communication linkbetween said speaker enclosure and said server; f. a visual displaysupported at least partially within said enclosure; g. an emergencyflasher supported at least partially within said enclosure; and h. akeyword database of emergency keywords and confirmation keywords withinan audio processor subsystem.
 12. The smart speaker system of claim 11,comprising said audio processor subsystem operable to: a. detectemergency key words, present in said emergency keyword database, in anoutput of said microphone; b. generate a confirmation message responsiveto detection of particular said emergency keywords; c. send an audiosignal of said confirmation message to said loudspeaker; d. detectconfirmation keywords in said output of said microphone; and e.communicate said confirmation keywords to a processor/driver.
 13. Thesmart speaker system of claim 12, wherein said audio processor subsystemcomprises at least one of hardware and software located in at least oneof said speaker enclosure and said server.
 14. The smart speaker systemof claim 12, wherein said processor/driver is operable, responsive toreceiving communication of said confirmation keywords and said emergencykeywords, to: a. generate an emergency notification message; b.determine appropriate clients to receive said emergency notificationmessage; c. communicate said emergency notification message via saidserver; d. drive said visual display responsive to reception of saidemergency notification message; e. drive said emergency flasherresponsive to reception of said emergency notification message; and f.drive said loudspeaker responsive to reception of said emergencynotification message.
 15. The smart speaker system of claim 13, whereinsaid server is operable to distribute said emergency notificationmessage to said appropriate clients via a communications network. 16.The smart speaker system of claim 13, wherein: a. said visual display isoperable to display at least one of text and symbolic images; b. saidemergency flasher is controllable to modulate intensity, color, andflash pattern, responsive to a type of emergency represented by saidemergency notification message; c. said loudspeaker is operable toaudibly present at least one of the emergency notification message, avoice response, and an alarm sound.
 17. The smart speaker system ofclaim 11, comprising noise cancellation circuitry operable to cancelloudspeaker output in said microphone output.
 18. A smart speaker systemcomprising: a. a speaker enclosure; b. a loudspeaker supported at leastpartially within said speaker enclosure c. a microphone supported withinsaid enclosure; d. a server operable to communicate over acommunication. e. a communication link between said speaker enclosureand said server; f. a visual display supported at least partially withinsaid enclosure; g. an emergency flasher supported at least partiallywithin said enclosure; and h. a keyword database of emergency keywordsand confirmation keywords within an audio processor subsystem; i. noisecancellation circuitry operable to cancel loudspeaker output in saidmicrophone output; and j. at least one of hardware and software locatedin at least one of said speaker enclosure and said server; k. whereinsaid audio processor subsystem operable to: i. detect emergency keywords, present in said emergency keyword database, in an output of saidmicrophone; ii. generate a confirmation message responsive to detectionof particular said emergency keywords; iii. send an audio signal of saidconfirmation message to said loudspeaker; iv. detect confirmationkeywords in said output of said microphone; and v. communicate saidconfirmation keywords to a processor/driver.
 19. The smart speakersystem of claim 18, wherein said processor/driver is operable,responsive to receiving communication of said confirmation keyword andsaid emergency keywords, to: a. generate an emergency notificationmessage; b. determine appropriate clients to receive said emergencynotification message; c. communicate said emergency notification messagevia said server; d. drive said visual display responsive to reception ofsaid emergency notification message; e. drive said emergency flasherresponsive to reception of said emergency notification message; and f.drive said loudspeaker responsive to reception of said emergencynotification message.
 20. The smart speaker system of claim 7, whereinsaid server is operable to distribute said emergency notificationmessage to said appropriate clients via a communications network.